Feature Learning for Conditional Random Fields and its Application to Gesture Recognition

Conditional random fields (CRFs) have been successful in many sequence labeling tasks, which conventionally rely on a hand-craft feature representation of input data. However, a powerful data representation could be another determining factor of the performance, which has not attracted enough attention yet. We describe a novel sequence labeling framework for gesture recognition, which builds a supervised CRF and an unsupervised dynamic model on a shared nonlinear feature transformation neural network. The model is a case of transfer learning that jointly optimizes two learning tasks together with learning a meaningful feature representation of input data. We demonstrate a gesture recognition system that yields a significant improvement of recognition accuracy over conventional CRFs.